Objectives

• Critically reflect on the what, why and how of evidence-based medicine
• Develop and practice your skills in critical appraisal and clinical epidemiology
• Be able to confidently interpret and apply clinical research in your practice

My short version…

EBM is a commitment to using best evidence to inform therapeutic decisions

Most quoted

Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.

Sackett et al. (1996)

Greenhalgh and Donald

EBM is the use of mathematical estimates of the chance of benefit and the risk of harm, derived from high-quality research on population samples, to inform clinical decision-making

Greenhalgh (2012)

There are many components to EBM

• Individual therapeutic decision-making
• Guidance on use of evidence: hierarchies of evidence; GRADE
• Critical appraisal
• Systematic review and meta-analysis methods
• Guideline development
• Systematic summaries of evidence
• Meta-research methods
• Clinician and patient decision aids
• Clinical epidemiology

Levels of evidence (OCEBM Levels of Evidence Working Group 2011)

GRADE

GRADE: Grading of Recommendations, Assessment, Development and Evaluation

GRADE offers a system for rating quality of evidence in systematic reviews and guidelines and grading strength of evidence of recommendations in guidelines

G. Guyatt et al. (2011) p. 384

GRADE Strength of Evidence

GRADE Strength of Recommendation

Factors in considering strength of recommendation

• Uncertainty about balance between desirable and undesirable effects
• Uncertainty or variability in values in preferences
• Uncertainty about whether the intervention represents a wise use of resources

G. H. Guyatt et al. (2008)

Flecainide

• Certain types of ventricular arrhythmia are a risk factor for death following myocardial infarction
• Prior to 1991, it was common to prescribe antiarrhythmic agents to post-MI patients with asymptomatic or mildly symptomatic ventricular arrhythmias (Morganroth, Bigger, and Anderson 1989)

CAST Trial (CAST Investigators 1991)

CAST: Outcome

Hormone Replacement Therapy

• From 1980–2000 many women were prescribed long-term hormone replacement therapy to reduce cardiovascular disease
• This practice was backed-up by what what was understood about the mechanisms and several large observational studies (Stampfer and Colditz 1991)

Women’s Health Initiative (Women’s Health Initiative Investigators 2003)

Women’s Health Initiative: Outcomes

• The Women’s Health Initiative study (2003) randomized 16,608 postmenopausal women to HRT or placebo and followed them for 5 years
• Overall hazard ratio for coronary heart disease 1.24 (95% unadjusted CI: 1.00–1.54)
• Women randomized to HRT suffered an additional 6 cases of coronary heart disease per 10,000 person years

How do we explain the very different results of the observational studies compared to WHI?

Study design

Confounder

Necessary conditions for a factor to be a confounding factor between exposure and outcome:

1. The factor must be an extraneous risk factor for the outcome
2. The factor must be associated with the exposure under study in the source population
3. The factor cannot be an intermediate step in the causal path between the exposure and the outcome

Life-course socioeconomic status and HRT

Key tools

1. Pre-appraised resources & systematic summaries
2. PICO
3. Critical appraisal & clinical epidemiology

Pre-appraised resources

• Cochrane Library
• BMJ Best Practice
• Trip Database
• DARE Database
• UptoDate
• Dynamed

PICO

Hypothesis testing

Underpowered tests

Overpowered tests

Interpreting results of tests with different power

Confidence intervals

95% Confidence interval

Ways to think about a confidence interval…

Precise, but confusing

• If the study was repeated many times, and the same procedure was used to calculate the 95% confidence interval, in the long run, you would expect the calculated 95% confidence intervals would include the true value of the parameter 95% of the time

• A 95% confidence interval provides the range of values that are not statistically different from the observed point estimate at the 0.05 level

Less precise, but useful

• The confidence interval provides a range of plausible values for the unknown parameter

• The lower limit is a likely lower bound estimate of the parameter; the upper limit a likely upper bound

Incorrect and misleading

• You can be 95% confident that the true value lies between the observed confidence interval

• The 95% confidence interval has a 95% chance of including the true effect size

Random error

What are your expectations when you toss a fair coin 10 times?

##  [1] 0 0 0 1 1 0 1 1 0 1

##  [1] 1 0 0 1 1 1 1 1 0 0

##  [1] 0 1 1 0 1 1 0 1 1 1

##  [1] 5 8 5 5 7 5 7 4 7 4

• Some 10 coin toss experiments provided 0 \(H\) or 10 \(H\)
• Most provided an outcome close to 5 \(H\)
• The mean number of \(H\) across all of these experiments was 4.998

Take home

1. Random error can make it hard to make inference from small samples of data.
2. Increasing the number of repetitions provided mean results much closer to what we expect given the coin is fair.

Task

Imagine you have coin of unknown bias (i.e. the probability of \(H\) is unknown—it is unknown whether the coin is fair, favours \(T\) or favours \(H\)). What test could you conduct to assess whether the coin is fair?

Attempt to describe the hypothesis you are testing and the statistical model you are using for the test.

Coin of unknown bias